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ABSTEACT 

The results of this study show that an experimental 
training program for graduate teacher assistants (TAs) affected the 
observable teaching behavior of the participants in two respects-- (1 ) 
participants made significantly greater use of instructional 
objectives and (2) they engaged in significantly more 

student-centered teaching. The subjects for this study were 19 novice 
TAs and 705 students enrolled in the introductory psychology course 
at Cornell University daring the fall of 1973. The independent 
variable of this experiment was defined as tA participation in either 
an experimental or a control version of a seminar entitled, "The 
Teaching of Psychology." At random, ten TAs were assigned to the 
experimental group, and the other nino TAs were assigned to the 
control group. The experimental version of the seminar consisted of 
scheduled reading assignments on coll 7e teaching, at least two 
individual conferences for each TA witu the course instructor, an 
individual video critique session for each TA, one unstructured group 
meeting with the course instructor^ and five formal workshop 
sessions. The training program was designed by the investigator to 
encompass a broad range of teaching competencies. Several general 
skills related to overall course design were emphasized: (1) 
specifying instructional objectives; (2) utilizing objectives in the 
design of instruction and tests; and (3) making the cognitive levels 
of objectives congruent with the cognitive levels of instructional 
activities and tests. Specific skills were emphasized for 
facilitating student-centered discussions. The control version of the 
seminar consisted of fewer and unscheduled reading assignments, two 
unstructured group meetings with the course instructor, and the 
opportunity for each TA to view a videotape of his or her teaching, 
but alone and without any critique. (MM) 
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As the adequacy of .training for college teachers has been 
criticized more and more widely during the past fifteen or 
twenty years, countless pre-service orientation programs, 
■in-service workshops, seminars, apprenticeship programs, intern 
programs, extern programs, and the like have been developed. 
During roughly the same period, the recruitment of teaching 
assistants (TAs) has been extended--f or both pedagogical and 
budgetary reasons --to the undergraduate level itself, and the 
use of undergraduate TAs may have helped, to heighten the per- 
ceived need for training as well as to increase the number and 
diversity of training programs prefssed into service. 

The recent development of TA training programs seems 
commendable, but it is surprising to note the lack of empirical 
research on the effects of such training. In the words of Barak 
Rosenshine, "We are instruction rich, and data poor" (1974, p. 4). 
Although the literature contains dozens of superficial descrip- 
tions of innovative training programs and numerous catalogs of 
materials for teacher training, only a handful of empirical 
studies used student performance and satisfaction measures to 
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assess the effectiveness of TA training programs. Most published 
reports either failed to provide evaluative data or merely pre- 
sented interview or questionnaire data from the TAs themselves. 

Only four studies have been identified that (1) included 
s.tudent variables among the criteria of effectiveness, (2) 
reported reliable observational data to assess whether differ- 
ences in training led to differences in specified teaching 
behaviors, and (3) utilized a true experimental design as 
defined by Campbell and Stanley (1963), All four involved 
training programs that focused on a few highly specific 
teaching competencies. Two of the four (Haber, 1973; Koffman, 
1974) failed to show any more than chance differences in the 
observed teaching behaviors or in student outcomes. The 
remaining two studies involved training in interaction 
analysis and heuristic questioning techniques only, and both 
were conducted with mathematics TAs. Training in interaction 
analysis alone was not very effective (Daniels, 1970), but 
training in both interaction analysis and heuristic questioning 
was effective (Tubb, 1974). 
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The results of four quasi-experimental studies (Cos tin, 
1968; Hockett, 1972; Lewis & Orvis, 1973; Yaghlian, 1972) 
generally indicated that training programs designed for a 
iwider scope of teaching competencies yielded significant 
effects on student achievement and student ratings of 
instruction. The training programs of these studies tended 
to span a range of topics such as course design, communica- 
tion, classroom climate, questioning, testing, influence 
styles, student participation, and the use of student feed- 
back. The validity of these quasi-experimental studies, 
hoxvever, was susceptible to the influence of history, matura- 
tion, and interaction effects, as set forth in Campbell and 
Stanley (1963). 

The present study with psychology TAs utilized a true 
experimental design to test whether a training program that 
was highly general in scope (1) affected the participants' 
teaching behavior, and (2) improved their student ratings of 
instruction. It was not feasible to measure the effective- 
ness of training in terms of student achievement. Since the 
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TAs of this sample each taught seminars on different topics 
within introductory psychology, there I'^as no basis for 
comparing the performance of students from different discus- 
sion sections. 

A third objective of the study, hox^ever, xvas to determine 
hoxv the TA training program affected the relationship betx^7een 
student ratings of instruction and student performance in the 
seminars. 

The independent variable of this experiment was a dichot- 
omous variable defined as TA participation in either an experi- 
mental or a control version of a seminar entitled "The Teaching 
of Psychology." Participation in the experimental version was 
mandatory for the ten TAs selected at random for the experi- 
mental group, and the control version was mandatory for the 
nine TAs randomly assigned to the control group. 

The experimental version consisted of scheduled reading 
assignments on college teaching, at least two individual 
conferences for each TA, an individual video critique session 
for each TA, one unstructured group meeting, and five formal 
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workshop sessions. This training program designed by the 
investigator to encompass a broad range of teaching competen- 
cies. Several skills related to overall course design x^/ere 
emphasized: specifying instructional objectives, utilizing 
objectives in the design of instruction and tests, and making 
the cognitive levels of objectives congruent xvith the cogni- 
tive levels of instructional activities and tests (Bloom, 1956) 
Analyzing the structure of knox^ledge x^hich underlies psycho- 
logical research documents (Gox^in, 1975) x^;as a subject matter 
competency that the training sought to enhance. Specific 
skills for facilitating student -centered discussions (McKeachie 
1969) included being an active listener, establishing a con- 
ducive classroom atmosphere, being aware of nonverbal communi- 
cation, obtaining feedback from group members, using varied and 
appropriate questioning techniques, and providing closure. 
Detailed documentation of the training procedures is given in 
Carroll (1976). 

The control version of the seminar consisted of fewer and 
unscheduled reading assignments on college teaching, two 
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unstructured group meetings with the course instructor, and the 
opportunity for each TA to view a videotape of his or her 
teaching alone and v7ithout any critique. During both meetings 
with the control group TAs, the course instructor had the 
participants share their own insights, but he purposely added 
few substantive conmients of his own. 

Seven dependent variables were defined and measured as 
f ollovjs . 

1. Use of objectives: the degree to which the TA speci- 
fied clearly formulated objectives and maintained agreement 
among objectives, instruction, and tests. 

2. Cognitive levels of questions: six mutually exclusive 
categories of questions as defined in Bloom (1956). The six 
levels, from lower order to higher order, are: memory, compre- 
hension, application, analysis, synthesis, and evaluation. 
This variable was measured by trained, independent raters who 
analyzed classroom questions and seminar quiz questions. 

3. Students-centered versus instructor-centered teaching: 
teaching characterized by accepting feelings, giving praise. 
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using students' ideas, and asking questions as opposed to 
lecturing, giving directions, and criticizing. This variable 
Xi7as measured by four trained, independent raters using the 
•Indirect/Direct ratio (l/D) from Flanders Interaction Analysis 
C1970) . 

4. Student talk ratio (S/T) : the ratio of minutes of 
student talk in class to minutes of student talk and TA talk 
in class, as measured by the number of tallies in the respec- 
tive Flanders categories, 

5. Student evaluation of TA effectiveness: the mean of 
student ratings on three questionnaire items related to 
teaching 

6. Student evaluation of seminar effectiveness: the mean 
of student ratings on four questionnaire items related to 
outcomes of the seminars. 

7. Student performance: numerical grade achieved by the 
student for overall seminar performance. The numerical grade 
V7as determined by a seminar quiz, a term paper, and class 
participation. 
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Hypotheses 

Seven hypotheses x^/ere formulated to predict differences 
bet\veen the experimental group and the control group. On the 
basis of previous quasi-experimental research, the hypotheses 
predict the expected direction of difference. The first four 
hypotheses preidict outcomes related to the first major objec- 
tive of the experiment, i.e., to determine x^hether the 
experimental TA training program affected the teaching behavior 
of the participants. 

1. The experimental group TAs x^7ill be rated higher than 
the control group TAs on the use of objectives. 

2. There x^;ill be greater congruity among the cognitive 
levels of classroom questions and quiz questions for the 
experimental group than for the control group. That is, the 
mean of the correlations between the frequency of each TA^s 
seminar questions at the various cognitive levels and the 
frequency of that TA*s quiz questions at the various cognitive 
levels x^;ill be higher for the experimental group as a x^hole 
than it xvill be for the control group. 
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3. The teaching of the experimental group TAs will be 
more student-centered than that of the control group TAs* 

4. The seminars taught by the experimental group TAs 
will have a higher student talk ratio than the seminars taught 
by the control group TAs. 

Hypotheses five and six predict outcomes related to the 
second major objective of the experiment, to determine whether 
the experimental training program improved the effectiveness 
of instruction as judged by the participants' students. 

5. The experimental group will have higher student 
evaluations of TA effectiveness than the control group. 

6. The experimental group will have higher student 
evaluations of. seminar effectiveness than the control group. 

Hypothesis seven addresses the third objective--to investi- 
gate whether the training program affected the correlation 
between student ratings and student performance in the partici* 
pants' seminars. 

7. For the students taught by the experimental group 
TAs, there will be a significant positive correlation between 
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student evaluations of TA effectiveness and student performance 
in seminar. 

The correlation coefficient for the control group could nop 
be predicted with any confidence. That coefficient could rea-- 
sbnably be expected to fall anyl^7here x^/ithin the x^ide range 
already reported in the literature. 

s 

Subj ects 

The subjects for this experiment x^/ere 19 novice TAs and 
705 students participating in the introductory psychology course 
at Cornell University during the fall semester of 1975. Four of 
the TAs were graduate students and fifteen were undergraduates. 
Six were male and thirteen were female. All TAs conducted 
weekly seminars in their own area of expertise as a supplement 
to the lectures given by the course instructor. The graduate 
TAs each met with four sections per week; the undergraduates 
each met with two sections per week. 

The TAs were matched on verbal aptitude, and ten were 
randomly assigned to the experimental group, nine to the 
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control group. There were no significant differences betv/^en 
the two groups of TAs on sex, grade level, verbal aptitude, 
cumulative average, or class size (p > .10). 

An analysis of student characteristics indicated that 
there were no significant differences between the experimental 
group and control group students on sex, college, grade level, 
approximate cumulative average, major, or primary reason for 
taking the course (p > .05). 

Procedure 

The experimental design was the posttest-only control 
group design (Campbell & Stanley, 1963). Since the novice TAs 
were assigned randomly to the experimental and control grasps 
and since. there were no significant differences in the TA 
characteristics and student characteristics reported earlier, 
it was assumed that there was no initial bias favoring one 
group of TAs over the other. The TAs were aware that their 
teaching was being studied for the purpose of group comparisons, 
but when questioned at the end of the semester, they could Hot 
identify specific variables being measured. 
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Observational data x^ere obtained hv \n'ri^otaping one 
class session of each TA during eriod near the 

end of the semester. The recordii.^^ ..i classes x^;as counter- 
balanced so that approximately the same number of experi- 
mental and control group TAs -were recorded each week and so 
that approximately the same number of each group were teaching 
for the first, second, third, or fourth time that week. 

Four trained raters independently analyzed each videotape 
using Flanders Interaction Analysis. Data on the cognitive 
levels of classroom questions and quiz questions were obtained 
from four other trained, independent raters using a modified 
version of the Teacher-Pupil Question Inventory (Davis & 
Tinsley, 1968). Table 1 contains the coefficients of inter- 
rater reliability. 

Table 1 



Inter-Rater Reliability 



Raters 


Coefficient 
of Agreement 


FIA 


7r = .84 


TPQI, classroom 


r = .73 


TPQI, quiz 


r = .74 
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usG of objectives, t (17) = 1.60, p <.07. Thus, hypothesis 
one \i7as confirmed. 

Table 2 

t-Test of the Use of Objectives 

Group Mean s « d . d£ t 

E 4.02 .44 

17 1.60 .07 

G 3.70 .43 

Hypothesis txvo stated that the experimental group TAs 
would achieve greater congruity among the cognitive levels of 
classroom questions and quiz questions than the control group 
TAs. A Pearson correlation coefficient x^as computed for each 
TA by pairing the number of classroom questions at each cog- 
nitive level \vith the number of quiz questions at that level. 
The mean coefficient for each group of TAs xvas then computed 
using the Fisher Z transformation, and a t-test xvas performed. 
As shown in Table 3, the difference between groups was not 
significant, t (17) = .90, p >.18. Thus, hypothesis two was 
rejected. 
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Table 3 
Congruity 

Group Mea r d. df t p 

E .^i 

17 .90 .18 

C .13 .70 

The Flanders data indicated that the teaching of the 

experimental group TAs was more student -centered than that of 

the control group TAs. As shown in Table 4, the l/D ratios of 

the experimental group were significantly hiiher than those of 

the control group, t (17) = 1.69, p <.06. Thus, the third 
hypothesis was confirmed. 

Table 4 



I/D Ratio 


Group 


Mean 


s.d. df 


t p 


E 


.44 


.21 








17 


1.69 .06 


C 


.31 


.11 





Hypothesis four, which predicted higher student talk 
ratios for the experimental group than for the control group. 
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V7as rejected. Table 5 shows that there was no significant 
difference between groups on the S/T ratio, t (17) = .36, 
P > .36. 

Table 5 



S/T Ratio 



Group Mean s . d . 


df 


t 


P . 


E .55 .20 








C * .52 .20 


17 


.36 


.36 



Hypotheses five and six stated that the stud^^^- evalua- 
tions of the experimental group xvouJ.d exceed tho5 f the 
control group f cr ZlA ef f ect i^jeness and seminar ef. tiveness, 
respectively. A- demonstrated iir Table 6, both hyc theses 
V7ere confirmed. 



Table 6 



Group 


Mean s.d. df t 


P 


TA e£f ecicr/eness 




4.19 .38 






17 1.34' 


.10 




3.98 .30 




Seminar effectiveness 




3.53 .58 






17 1.36 


.10 


C 


3.18 .52 
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Students rated the effectiveness of the experimental 
group TAs significantly higher than that of the control group 
TAs, t (17) = 1.34 p <.10. Students also rated the seminars 
taught by the experimental group TAs as being significantly 
mpre effective than the semlnni^i taught by controls, t (17) = 
1.36, p <.10. 

In addition, as shown in Table 7, the overall Pearson 
correlation between the l/D ratios and the mean ratings of TA 
effectiveness was pcs"-^-^, r = .53, p < .02, two-tailed. 
Likewise, the correl^; ion :32etween l/D ratios and the mean 
ratings of seminar efi ;cxi-rsness was positive, r = .57, p <.02, 
two-tailed. Therefore «as concluded that the use of 

indirect teaching was ta^ir£±5rely associated with student 
ratings of the effec- :?se~s5 of instruction. 



Table 7 


Var- ,.(3; = 




r 


_ P. 


I/D & TA efrz^ 


iveness 


.53 


.023 


I/D 6c seminsr 


effectiveness 


.57 


,02a 



^The s±% hi — c ance level is tvo-tailed. 
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As indicated in Table 8, student ratings of TA effectiveness 
shox^ed no significant difference between the nine TAs x^/ith the 
greatest congruity and the nine x^ith the least congruity among 
cognitive levels of classroom and quiz questions, t (16) = .58, 
p-> .57, two-tailed. Similarly, there was no significant dif- 
ference between the high- and low-congruity groups on student 
ratings of seminar effectiveness, t (16) = .54, p > .59, two- 
tailed. Thus, there was no evidence of an association between 
the congruity variable and student ratings of instruction. 





Table 8 




High 


- vs. Low-Congruity Group 


s 


Group 


Mean s.d. df t 


P 


TA effectiveness 


High 
Low 


4 . 14 .34 

16 .58 

4.04 .40 


.57^ 


Seminar effectiveness 


High 
Low 


3.42 .62 

16 . 54 

3.27 .54 


.59^ 



^The significance level is 
two-tailed. 
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Finally, hypothesis seven predicted that for the experi- 
mental group there x^;ould be a significant positive correlation 
bet\veen student ratings of TA effectiveness and student per- 
formance in seminar. This correlation xvas computed by pairing 
t'he mean seminar grade of each seminar section x^/ith the mean 
rating of TA effectiveness from that section. The results are 
presented in Table 9. There is no evidence that the training 
program significantly increased the correlation between student 
perxrormance and student rat in2:s of either TA effectiveness or 
seminar effectiveness • 

rable 9 

Correlations x^;rtrr Student Performance 



Group r p 

TA effectiveness 

E .21 .15 

C .13 .59^ 



5 eminar e f f e c t ivene s s 
E .18 .37a 

C .15 .53^ 



^The significance level is txvo- 
nailed. 
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For the nine high-congruity TAs, however, there was a 
significant correlation between student performance in seminar 
and student ratings of TA effectiveness, as shown in Table 10 
(r = .38, p <.08, two-tailed). This correlation coefficient 
for the high-congruity TAs was significantly greater than the 
corresponding coefficient (r = -.19}) for the nine low-congruity 
TAs (p <.08, two-tailed). As for the correlation between 
student performance and student ratings of seminar effective- 
ness, the coefficients were not significant for either the 
high-congruity group (r = .24, p > .J.5, two -tailed) or the 
IxDw-congruity group (r = -.12, p > .£1, two-tailed), and the two 
coefficients were not significantly different from each other 
(p > .25, two-tailed) . 
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Table 10 

Correlations with leiiL rex foxniauce 
(High- and Low-Congruity Groups) 



Group r p 

TA effectiveness 

High .38 .08^ 

Low -.19 .42^ 



Seminar effectiveness 
High .24 .25^ 
Low - . 12 .61^ 



^The significance level is tv^o- 
tailed. 

The significant difference between the high- and 
low-congruity groups on the correlation between student per- 
formance and student ratings of TA effectiveness was inter- 
preted as suggesting that the extreme inconsistency among 
published reports of such correlations (reviewed in Kulik & 
Kulik, 1974) may be attributable to inappropriate measures of 
student preformance. These data suggest that the variability 
in reported correlation coefficients may be distorted by the 
variability from one study to another in the degree to which 
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the measures of performance \^7ere congruent vith course objec- 
tives and instructional tasks. 

In sum, four general conclusions can be stated, 
1. The experimental TA training program affected the 
observable teaching behavior of the participants in tx^7o 
respects. The participants made significantly greater use of 
objectives (p <.07), and they engaged in significantly more 
student -centered teaching (p <,05). 

1. Student ratings of the effectiveness of instruction 
were significantly higher for the participants than for the 
controls (p < . 10) . 

3. Regardless of training, the use of indirect teaching 
skills -was positively correlated x^;ith student ratings of 
instructional effectiveness (p<,02, txvo-tailed) . 

4. Regardless of training, for the nine TAs x^ith the 
greatest congruity among the cognitive levels of classroom 
and quiz questions, as compared tD rhe nine xvith the least 
congruity, there x^;as a significantly higher correlation 
betx^;een student ratings of TA effectiveness and student 
performance in seminar (p <.08, tx^;o-tailed) . 
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